Integration of Riemannian Motion Policy and Whole-Body Control for Dynamic Legged Locomotion

In this paper, we present a novel Riemannian Motion Policy (RMP)flow-based whole-body control framework for improved dynamic legged locomotion. RMPflow is a differential geometry-inspired algorithm for fusing multiple task-space policies (RMPs) into a configuration space policy in a geometrically consistent manner. RMP-based approaches are especially suited for designing simultaneous tracking and collision avoidance behaviors and have been successfully deployed on serial manipulators. However, one caveat of RMPflow is that it is designed with fully actuated systems in mind. In this work, we, for the first time, extend it to the domain of dynamic-legged systems, which have unforgiving under-actuation and limited control input. Thorough push recovery experiments are conducted in simulation to validate the overall framework. We show that expanding the valid stepping region with an RMP-based collision-avoidance swing leg controller improves balance robustness against external disturbances by up to $53\%$ compared to a baseline approach using a restricted stepping region. Furthermore, a point-foot biped robot is purpose-built for experimental studies of dynamic biped locomotion. A preliminary unassisted in-place stepping experiment is conducted to show the viability of the control framework and hardware

A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems

Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to "real world" events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and systems will instead need to adapt to novel distributions and tasks while deployed. This critical gap may be addressed through the development of "Lifelong Learning" systems that are capable of 1) Continuous Learning, 2) Transfer and Adaptation, and 3) Scalability. Unfortunately, efforts to improve these capabilities are typically treated as distinct areas of research that are assessed independently, without regard to the impact of each separate capability on other aspects of the system. We instead propose a holistic approach, using a suite of metrics and an evaluation framework to assess Lifelong Learning in a principled way that is agnostic to specific domains or system techniques. Through five case studies, we show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems. We highlight how the proposed suite of metrics quantifies performance trade-offs present during Lifelong Learning system development - both the widely discussed Stability-Plasticity dilemma and the newly proposed relationship between Sample Efficient and Robust Learning. Further, we make recommendations for the formulation and use of metrics to guide the continuing development of Lifelong Learning systems and assess their progress in the future.Comment: To appear in Neural Network

Q-functionals for Value-Based Continuous Control

We present Q-functionals, an alternative architecture for continuous control deep reinforcement learning. Instead of returning a single value for a state-action pair, our network transforms a state into a function that can be rapidly evaluated in parallel for many actions, allowing us to efficiently choose high-value actions through sampling. This contrasts with the typical architecture of off-policy continuous control, where a policy network is trained for the sole purpose of selecting actions from the Q-function. We represent our action-dependent Q-function as a weighted sum of basis functions (Fourier, Polynomial, etc) over the action space, where the weights are state-dependent and output by the Q-functional network. Fast sampling makes practical a variety of techniques that require Monte-Carlo integration over Q-functions, and enables action-selection strategies besides simple value-maximization. We characterize our framework, describe various implementations of Q-functionals, and demonstrate strong performance on a suite of continuous control tasks

Suppression Mechanism of TiO2 for the Partial Discharge of Oil-paper Insulation in Intensive Electric Field

With the rapid development of modern HVDC transmission technology, higher insulation properties are put forward on the oil-paper insulation system of the transformer, which determine the transformer service life to a certain extent. Traditional transformer oil-paper insulation is becoming increasingly difficult to meet the demands of insulation system with large capacity and miniaturization at ultra-high voltage level. In order to improve the insulation strength of oil-paper system, the insulation cellulose paper modified by TiO2 nanoparticles of different diameters (5 nm, 10 nm, 20 nm, 30 nm) were prepared, in addition, each of modified cellulose paper has different mass fraction of TiO2 nanoparticles (1%, 3%, 5%, 7% wt.). The partial discharge (PD) detection platform was established, and the partial discharge inception voltage (PDIV) values of the oil-paper insulation system with and without nanoparticles were measured. To investigate the PD characteristics, the PD waveforms and PD frequency spectrums of modified cellulose paper and the unmodified were obtained. The suppression mechanism of TiO2 nanoparticles on PD was explored through scanning electron microscope (SEM) observation. All the experiment results indicate that adding nano-TiO2 is beneficial to enhance the insulation properties of oil-paper insulation, and the optimum diameter and mass fraction of TiO2 nanoparticles to suppress oil-paper PD were obtained

Comprehensive Carbon Emission and Economic Analysis on Nearly Zero-Energy Buildings in Different Regions of China

Considering the comprehensive effect of building carbon emissions, cost savings is of great significance in nearly-zero-energy buildings (NZEBs). Previous research mostly focused on studying the impact of technical measures in pilot projects. The characteristics of different cities or climate zones have only been considered in a few studies, and the selection of cities is often limited. At times, only one city is considered in each climate zone. Therefore, this study selected 15 cities to better cover climate zone characteristics according to the variation in weather and solar radiation conditions. A pilot NZEB project was chosen as the research subject, in which the energy consumption was monitored and compared across different categories using simulated values by EnergyPlus software. Various NZEB technologies were considered, such as the high-performance building envelope, the fresh air heat recovery unit (FAHRU), demand-controlled ventilation (DCV), a high-efficiency HVAC and lighting system, daylighting, and photovoltaic (PV). The simulated carbon emission intensities in severe cold, cold, and hot summer and cold winter (HSCW) climate zones were 21.97 kgCO2/m2, 19.60 kgCO2/m2, and 15.40 kgCO2/m2, respectively. The combined use of various NZEB technologies resulted in incremental costs of 998.86 CNY/m2, 870.61 CNY/m2, and 656.58 CNY/m2. The results indicated that the HSCW region had the best carbon emission reduction potential and cost-effectiveness when adopting NZEB strategies. Although the incremental cost of passive strategies produced by the envelope system is higher than active strategies produced by the HVAC system and lighting system, the effect of reducing the building’s heating load is a primary and urgent concern. The findings may provide a reference for similar buildings in different climate zones worldwide

Comprehensive Carbon Emission and Economic Analysis on Nearly Zero-Energy Buildings in Different Regions of China

Considering the comprehensive effect of building carbon emissions, cost savings is of great significance in nearly-zero-energy buildings (NZEBs). Previous research mostly focused on studying the impact of technical measures in pilot projects. The characteristics of different cities or climate zones have only been considered in a few studies, and the selection of cities is often limited. At times, only one city is considered in each climate zone. Therefore, this study selected 15 cities to better cover climate zone characteristics according to the variation in weather and solar radiation conditions. A pilot NZEB project was chosen as the research subject, in which the energy consumption was monitored and compared across different categories using simulated values by EnergyPlus software. Various NZEB technologies were considered, such as the high-performance building envelope, the fresh air heat recovery unit (FAHRU), demand-controlled ventilation (DCV), a high-efficiency HVAC and lighting system, daylighting, and photovoltaic (PV). The simulated carbon emission intensities in severe cold, cold, and hot summer and cold winter (HSCW) climate zones were 21.97 kgCO2/m2, 19.60 kgCO2/m2, and 15.40 kgCO2/m2, respectively. The combined use of various NZEB technologies resulted in incremental costs of 998.86 CNY/m2, 870.61 CNY/m2, and 656.58 CNY/m2. The results indicated that the HSCW region had the best carbon emission reduction potential and cost-effectiveness when adopting NZEB strategies. Although the incremental cost of passive strategies produced by the envelope system is higher than active strategies produced by the HVAC system and lighting system, the effect of reducing the building’s heating load is a primary and urgent concern. The findings may provide a reference for similar buildings in different climate zones worldwide

A domain-agnostic approach for characterization of lifelong learning systems

Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to “real world” events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and systems will instead need to adapt to novel distributions and tasks while deployed. This critical gap may be addressed through the development of “Lifelong Learning” systems that are capable of (1) Continuous Learning, (2) Transfer and Adaptation, and (3) Scalability. Unfortunately, efforts to improve these capabilities are typically treated as distinct areas of research that are assessed independently, without regard to the impact of each separate capability on other aspects of the system. We instead propose a holistic approach, using a suite of metrics and an evaluation framework to assess Lifelong Learning in a principled way that is agnostic to specific domains or system techniques. Through five case studies, we show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems. We highlight how the proposed suite of metrics quantifies performance trade-offs present during Lifelong Learning system development — both the widely discussed Stability-Plasticity dilemma and the newly proposed relationship between Sample Efficient and Robust Learning. Further, we make recommendations for the formulation and use of metrics to guide the continuing development of Lifelong Learning systems and assess their progress in the future